Business problem¶
In this project we will try to find the the right location for a coffee shop in London (understanding location as Borough or cluster of Boroughs). The criteria to classify and rank the different Boroughs is agreed with the client.
Demographic preferences.
- Growing population
- Working population
- Mid-high income
- Low crime rate
Analytics approach¶
We are going to search for different sources of data that combined can help us make a better decision. The data we are going to analyze can be grouped into:
- Pure demographics data. Looking at areas where the population is growing.
- Profiling data including unemployment rates, average gross income and average house price and crime indicators.
- Business metrics, number of businesses, two-year business survival rate.
- Competition and other business in the area information.
Data Sources¶
London Boroughs Population - UK National Statistics
London Borough Profiles - London Data Store
London Boroughs GeoJson file - Carto.com
FourSquare API - Google Cloud
Postcode Lookup API - Postcodes.io
In [6]:
df_web.head()
Out[6]:
1.2 London Borough Profiling data¶
The London Borough Profiles help paint a general picture of an area by presenting a range of headline indicator data covering demographic, economic, social and environmental datasets for each borough, alongside relevant comparator areas. We are taking only the variables we consider relevant for this project.
In [9]:
df_pro.head()
Out[9]:
1.3 GeoJson data¶
GeoJSON is a file format for representing geodata as JSON. In this format, every Feature element has a geometry (in this case, the geometry of the different boroughs) and additional properties.
In [12]:
df_geo.head()
Out[12]:
1.4 Competition and other businesses¶
We are using the Foursquare API to get information about the surrounding areas.
In [14]:
location['lat'] = location.geometry.centroid.y
location['lng'] = location.geometry.centroid.x
location = location.drop('geometry', axis=1)
location.head()
Out[14]:
In [17]:
boroughs_venues = pd.read_csv('data/venues_from_foursquare.csv', index_col=0)
boroughs_venues.head()
Out[17]:
In [18]:
print(boroughs_venues['venue_cat'].value_counts())
boroughs_venues.head()
Out[18]:
In [32]:
venues_grouped = boroughs_venues_filtered.groupby('borough')['venue_cat'].value_counts()
venues_grouped
Out[32]:
In [35]:
# we merge the cafe and coffee shop columns into one
venues_grouped_unst['Cafes'] = venues_grouped_unst['Coffee Shop'] + venues_grouped_unst['Café']
venues_grouped_unst = venues_grouped_unst.drop(['Coffee Shop', 'Café'], axis=1)
venues_grouped_unst.head()
Out[35]:
1.5 Merging the Data¶
We are going to combine all dataframes into one. We have df_web, df_pro, df_geo and venues_grouped_unst
In [37]:
df = df_geo.merge(right=df_concat, left_on=df_geo.name, right_on=df_concat.index)
df = df.drop('key_0', axis=1)
print(type(df))
df.head()
Out[37]:
2. Data Exploration¶
Let's look at the data we have gathered to try to identify good locations for a new business opportunity.
In [25]:
# we summarise the different variables
df.describe().T
Out[25]:
In [26]:
# we deal with null values
df.isnull().sum()
Out[26]:
In [38]:
# we subset all the numeric variables to perform further analysis
X = df[['pop_17', 'pop_inc', 'inner_outer', 'percent_pop_work_age',
'unemployment_rate', 'gross_annual_pay',
'active_businesses', 'two_year_survival_rate',
'crime_rate_per_thousand', 'median_house_price',
'Hotel', 'Park', 'Pub', 'Cafes']]
In [39]:
# we create a correlation matrix to identify existing relationships in the variables
corrMatt = X.corr()
mask = np.array(corrMatt)
mask[np.tril_indices_from(mask)] = False
fig,ax= plt.subplots()
fig.set_size_inches(20,10)
sns.heatmap(corrMatt, mask=mask,vmax=.8, cmap='YlGnBu', linewidths=.5, square=True,annot=True)
Out[39]:
Insight: There matrix shows some interesting correlations.
Crime rate and active businesses.
In [41]:
# we convert categorical column with get_dummies
X = pd.get_dummies(X, drop_first=True)
X.head()
Out[41]:
In [ ]:
clt_labels = model.labels_
clt_labels
Clusters: We choose 6 clusters for the model to minimize the distortion score
In [45]:
from yellowbrick.cluster import KElbowVisualizer
# Instantiate the visualizer
visualizer = KElbowVisualizer(model, k=(4,12))
visualizer.fit(X_std) # Fit the data to the visualizer
visualizer.poof() # Draw/show/poof the data
In [80]:
from yellowbrick.cluster import InterclusterDistance
visualizer = InterclusterDistance(KMeans(5))
visualizer.fit(X_std) # Fit the training data to the visualizer
visualizer.poof() # Draw/show/poof the data
3.2 Comparing the clusters¶
We can start comparing the different clusters using the criteria agreed at the begining of the project.
In [52]:
print('Cluster 0 is formed by', len(df_labeled[df_labeled['cluster_label'] == 0]), 'Boroughs:', df_labeled[df_labeled['cluster_label'] == 0]['name'].str.cat(sep=', '))
print('Cluster 1 is formed by', len(df_labeled[df_labeled['cluster_label'] == 1]), 'Boroughs:', df_labeled[df_labeled['cluster_label'] == 1]['name'].str.cat(sep=', '))
print('Cluster 2 is formed by', len(df_labeled[df_labeled['cluster_label'] == 2]), 'Boroughs:', df_labeled[df_labeled['cluster_label'] == 2]['name'].str.cat(sep=', '))
print('Cluster 3 is formed by', len(df_labeled[df_labeled['cluster_label'] == 3]), 'Borough:', df_labeled[df_labeled['cluster_label'] == 3]['name'].str.cat(sep=', '))
print('Cluster 4 is formed by', len(df_labeled[df_labeled['cluster_label'] == 4]), 'Boroughs:', df_labeled[df_labeled['cluster_label'] == 4]['name'].str.cat(sep=', '))
print('Cluster 5 is formed by', len(df_labeled[df_labeled['cluster_label'] == 5]), 'Borough:', df_labeled[df_labeled['cluster_label'] == 5]['name'].str.cat(sep=', '))
Clusters: These are the resulting clusters with the boroughs that form them
We can quickly see the boroughs on a map
In [110]:
lat = 51.510067
lng = -0.133869
zoom_start = 10
m = folium.Map(location=[lat, lng], zoom_start=zoom_start) # generate map centred around the Conrad Hotel
ft = 'cluster_label'
cmap = cm.linear.viridis.scale(df[ft].min(), df[ft].max())
folium.GeoJson(df, style_function=lambda feature: {
'fillColor': cmap(feature['properties'][ft]),
'fillOpacity' : 0.7,
'weight' : 0.4,
'color' : 'black'
}).add_to(m)
marker = MarkerCluster().add_to(m)
for ix, row in df.iterrows():
text = "Borough: " + str(row['name']) + "<br>" + ft + ': '+ str(row[ft]) + "<br>" + "Population: " + str(row['pop_17'])
popup = folium.Popup(IFrame(text, width=300, height=100))
folium.Marker(location = [row.geometry.centroid.y ,row.geometry.centroid.x], popup=popup).add_to(marker)
cmap.caption = 'Number of Cafes'
m.add_child(cmap)
m
Out[110]:
In [54]:
df_cluster
Out[54]:
3.2.1 Growing population¶
Which cluster shows higher growth rates?
In [93]:
pop = sns.barplot(x='cluster_label', y='pop_inc', data=df_cluster)
pop.set_title('Population Increment')
pop.set_xlabel('Clusters')
pop.set_ylabel('Population Increment')
sns.despine(left=True)
3.2.2 Working population¶
Which cluster show lower unemployment rates?
In [98]:
pop = sns.barplot(x='cluster_label', y='unemployment_rate', data=df_cluster)
pop.set_title('Unemployment rate')
pop.set_xlabel('Clusters')
pop.set_ylabel('Unemployment rate')
sns.despine(left=True)
3.2.3 Mid-high income¶
Which cluster show higher gross annual income?
In [96]:
pop = sns.barplot(x='cluster_label', y='gross_annual_pay', data=df_cluster)
pop.set_title('Gross annual pay')
pop.set_xlabel('Clusters')
pop.set_ylabel('Gross annual pay')
sns.despine(left=True)
3.2.4 Crime rates¶
Which cluster show lower crime rates?
In [94]:
pop = sns.barplot(x='cluster_label', y='crime_rate_per_thousand', data=df_cluster)
pop.set_title('Crimes per thousand')
pop.set_xlabel('Clusters')
pop.set_ylabel('Crimes per thousand')
sns.despine(left=True)
3.2.5 Business survival rates?¶
Which cluster show higher survival rate for businesses after two years of activity?
In [95]:
pop = sns.barplot(x='cluster_label', y='two_year_survival_rate', data=df_cluster)
pop.set_title('Survival rate')
pop.set_xlabel('Clusters')
pop.set_ylabel('Two-year survival rate')
sns.despine(left=True)
3.3 What cluster fits the requirements?¶
Result Once analysed we would recommend opening the business in cluster 4
4. Building Visualizations¶
4.1 What boroughs are growing in population?¶
In [100]:
lat = 51.510067
lng = -0.133869
zoom_start = 10
avg_inc = np.mean(df.pop_inc)
m = folium.Map(location=[lat, lng], zoom_start=zoom_start) # generate map centred around the Conrad Hotel
ft = 'pop_inc'
cmap = cm.linear.viridis.scale(df[ft].min(), df[ft].max())
folium.GeoJson(df, style_function=lambda feature: {
'fillColor': cmap(feature['properties'][ft]),
'fillOpacity' : 0.5,
'weight' : 0.4,
'color' : 'black'
}).add_to(m)
for row in df.itertuples():
m.add_child(folium.Marker(location=[row.geometry.centroid.y, row.geometry.centroid.x],
popup=row.name, icon=folium.Icon(color='green' if row.pop_inc > avg_inc else 'purple', prefix='fa', icon='circle')))
cmap.caption = 'Population Increase'
m.add_child(cmap)
legend_html = '''
<div style="position: fixed;
bottom: 50px; left: 50px; width: 130px; height: 90px;
border:2px solid grey; z-index:9999; font-size:14px; background-color:white;
"> Population growth <br>
Above avg <i class="fa fa-map-marker fa-2x" style="color:green"></i><br>
Below avg <i class="fa fa-map-marker fa-2x" style="color:purple"></i>
</div>
'''
m.get_root().html.add_child(folium.Element(legend_html))
m
Out[100]:
4.2 What boroughs show higher business survival rates?¶
In [55]:
lat = 51.510067
lng = -0.133869
zoom_start = 10
m = folium.Map(location=[lat, lng], zoom_start=zoom_start) # generate map centred around the Conrad Hotel
ft = 'two_year_survival_rate'
cmap = cm.linear.viridis.scale(df[ft].min(), df[ft].max())
folium.GeoJson(df, style_function=lambda feature: {
'fillColor': cmap(feature['properties'][ft]),
'fillOpacity' : 0.7,
'weight' : 0.4,
'color' : 'black'
}).add_to(m)
marker = MarkerCluster().add_to(m)
for ix, row in df.iterrows():
text = "Borough: " + str(row['name']) + "<br>" + ft + ': '+ str(row[ft]) + "<br>" + "Population: " + str(row['pop_17'])
popup = folium.Popup(IFrame(text, width=300, height=100))
folium.Marker(location = [row.geometry.centroid.y ,row.geometry.centroid.x], popup=popup).add_to(marker)
cmap.caption = 'Two-year survival rate for businesses'
m.add_child(cmap)
m
Out[55]:
4.3 What boroughs show higher crime rates?¶
In [56]:
lat = 51.510067
lng = -0.133869
zoom_start = 10
m = folium.Map(location=[lat, lng], zoom_start=zoom_start) # generate map centred around the Conrad Hotel
ft = 'crime_rate_per_thousand'
cmap = cm.linear.YlOrBr_09.scale(df[ft].min(), df[ft].max())
folium.GeoJson(df, style_function=lambda feature: {
'fillColor': cmap(feature['properties'][ft]),
'fillOpacity' : 0.7,
'weight' : 0.4,
'color' : 'black'
}).add_to(m)
marker = MarkerCluster().add_to(m)
for ix, row in df.iterrows():
text = "Borough: " + str(row['name']) + "<br>" + ft + ': '+ str(row[ft]) + "<br>" + "Population: " + str(row['pop_17'])
popup = folium.Popup(IFrame(text, width=300, height=100))
folium.Marker(location = [row.geometry.centroid.y ,row.geometry.centroid.x], popup=popup).add_to(marker)
cmap.caption = 'Crime Rate Per 1000'
m.add_child(cmap)
m
Out[56]:
4.4 What boroughs have more cafes?¶
In [57]:
lat = 51.510067
lng = -0.133869
zoom_start = 10
m = folium.Map(location=[lat, lng], zoom_start=zoom_start) # generate map centred around the Conrad Hotel
ft = 'Cafes'
cmap = cm.linear.viridis.scale(df[ft].min(), df[ft].max())
folium.GeoJson(df, style_function=lambda feature: {
'fillColor': cmap(feature['properties'][ft]),
'fillOpacity' : 0.7,
'weight' : 0.4,
'color' : 'black'
}).add_to(m)
marker = MarkerCluster().add_to(m)
for ix, row in df.iterrows():
text = "Borough: " + str(row['name']) + "<br>" + ft + ': '+ str(row[ft]) + "<br>" + "Population: " + str(row['pop_17'])
popup = folium.Popup(IFrame(text, width=300, height=100))
folium.Marker(location = [row.geometry.centroid.y ,row.geometry.centroid.x], popup=popup).add_to(marker)
cmap.caption = 'Number of Cafes'
m.add_child(cmap)
m
Out[57]:
4.5 What boroughs show lower unemployment rate?¶
In [58]:
lat = 51.510067
lng = -0.133869
zoom_start = 10
m = folium.Map(location=[lat, lng], zoom_start=zoom_start) # generate map centred around the Conrad Hotel
ft = 'unemployment_rate'
cmap = cm.linear.viridis.scale(df[ft].min(), df[ft].max())
folium.GeoJson(df, style_function=lambda feature: {
'fillColor': cmap(feature['properties'][ft]),
'fillOpacity' : 0.7,
'weight' : 0.4,
'color' : 'black'
}).add_to(m)
marker = MarkerCluster().add_to(m)
for ix, row in df.iterrows():
text = "Borough: " + str(row['name']) + "<br>" + ft + ': '+ str(row[ft]) + "<br>" + "Population: " + str(row['pop_17'])
popup = folium.Popup(IFrame(text, width=300, height=100))
folium.Marker(location = [row.geometry.centroid.y ,row.geometry.centroid.x], popup=popup).add_to(marker)
cmap.caption = 'Unemployment rate'
m.add_child(cmap)
m
Out[58]:
In [59]:
lat = 51.510067
lng = -0.133869
zoom_start = 10
m = folium.Map(location=[lat, lng], zoom_start=zoom_start) # generate map centred around the Conrad Hotel
ft = 'active_businesses'
cmap = cm.linear.viridis.scale(df[ft].min(), df[ft].max())
folium.GeoJson(df, style_function=lambda feature: {
'fillColor': cmap(feature['properties'][ft]),
'fillOpacity' : 0.7,
'weight' : 0.4,
'color' : 'black'
}).add_to(m)
marker = MarkerCluster().add_to(m)
for ix, row in df.iterrows():
text = "Borough: " + str(row['name']) + "<br>" + ft + ': '+ str(row[ft]) + "<br>" + "Population: " + str(row['pop_17'])
popup = folium.Popup(IFrame(text, width=300, height=100))
folium.Marker(location = [row.geometry.centroid.y ,row.geometry.centroid.x], popup=popup).add_to(marker)
cmap.caption = 'Number of Active Businesses'
m.add_child(cmap)
m
Out[59]: